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Abstract 

This paper describes a series of collaborative research seminars designed to aliow classroom teachers 
to observe and assess critical aspocts of Logo learning in their own classrooms. We define critical 
aspects of Logo to be those concepts drawn from the domains of computer science, mathematical 
thinking and problem-solving which are essential for students to use and understand if they are to gain 
maximum benefit from their Logo learning experiences. This paper describes the approaches we 
developed to »upport classroom teachers in observing their own students, collecting data about their 
work, and assessing their learning of these aspects of Logo. We also describe our approach to assessing 
the professional development of teachers who participated with us as collaborative researchers. 

This work was supported by the National Science Foundation as part of Exploratory Research on 
Critical Aspects of Logo Leerning, grant number MDR 865 1600. We would also like to 
acknowledge the contributions of research consultants Roy Pea of New York University and Jan Hawkins 
of Bank Street College of Education, our Research Assistant, Joan Funk and Administrative Assistant, 
Gerry Sills of Education Development Center, and collaborating teacher researchers, Elizabeth Berrner, 
David Crump, Diana Freedman, Doreen Kelly, Robert Kondel, Jane Manzelli, Mary Miller-Teehan. 
Pasquale Puleo and Kitsy Rothermol. Any ideas, opinions, findings and conclusions expressed in this 
paper are those of the authors and do not neccessarily reflect the views of the National Science 
Foundation or of any of our collaborators. 
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I. INTRODUCTION AND BACKGROUND 

Our project started with questions by teacherc and findings from research which indicate that 
the educational potential of Logo is not yet lully realized in clissrooms. We are often told, "Logo 
isn't working," or "Students aren't learning in portant mathematics and computer science 
through exploration and discovery." (See, for example: Pea, et. aL Kurland et. aL 1985; 
Watt, D. 1 982; Moursund, 1 983; Leron, 1 985a, 1 985b) These opinions, distressing as they 
sound, match some of our own observations while teaching Logo to both children and teachers. 
The learning challenges our own students encounter demonstrate the types of difficulties 
experienced by many Logo learners. 



As we compared these observations about student learning with the powerful ideas embedded into 
the Logo language itself by its developers (Papert, 1980' Watt D, 1982; Watt M. 1982; Watt 
and Watt, 1986), we began to develop the concept of critical aspects of Logo learning, aspects of 
Logo which when understood and used by a Logo programmer allow that person to use Logo 
powerfully. And in our experience, the most powerful uses of Logo involve project-oriented 
work of the learner's own choosing which leads to the learning of important mathematical and 
computer science ideas while in the process of carrying out the project. Critical aspects, 
therefore, can serve as a framework by which a teacher can assess and guide students as thfjy 
work on their own projects. 

For the past year we have been working with teachers as collaborative researchers to use the 
concept of critical aspects of Logo to identify and describe important Logo learning in their own 
classrooms. It was our expectation that teachers who participated with us in this work would be 
able to document and demonstrate the value of the Logo learning in their own clas srooms. To 
support this process we conducted an ogoing series of seminars to involve the teachers in a 
community of colleagues with a shared purpose. 

Part of our model for collaborative research seminars came from our own experiences as 
teachers and researchers. Our extensive work with the development of teacher centers 
encouraged us to value teachers' professionalism. We modeled some of our seminar structure on 
the Children's Thinking (Research) Seminars developed and led by Bill Hull and Sara Hull (Hull, 
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1979; Watt. M, 1979). Some of ou^ ideas about supporting teacher investigations were 
learned from Evans, Bamberger and Duckworth (Evans, Stubbs, Duckworth, and Davis, 1981 ; 
Bamberger. Duckworth, and Lampert. 1981 ; Duckworth. 1986). In our research and 
development work with Patricia Carini at The Prospect Archive of Children's Work and Research 
Center. North Bennington, Vermont, we participated in developing procedures for teachers to 
si'pport each others' classroom action research projects (Carini, 1979). Our participation in 
The North Dakota Study Group on Evaluation gives us many models of qualitative research, 
observation and documentation processes teachers can carry out in their own classrooms. 

This collaborative research project embraces many of our concerns for educational reform. We 
desire to support teachers in developing a growing sense of their own- professionalism. And v/e 
desire to support both teachers and students in using Logo in the powerful ways and for the 
powerful purposes envisioned by its developers (Papert, 1980). In this paper we focus on the 
processes and experiences of the teacher research seminars and on the professional development 
of participating teachers. Issues related to critical aspects of Logo, and student learning will be 
discussed elsewhere. 



II. THE RESEARCH SEMINARS 

Our design for a teacher research seminar was based on prior experience with collaborative 
research, and on a broad survey of research on professional development of practicing teachers. 
Of particular importance to us is the research on Stages of Concern, and the Concerns-Based 
Adoptfon Model (Hall, et. al, 1973; Loucks and Hall. 1979). The central finding of this work is 
that teachers e^igaged in implementing a classroom innovation go through a predictable 
development in their concerns about and attitudes towards that infio\'ation. Teacher education 
efforts that take these concerns into account, provide opportunities for teachers to express 
their concerns, and adapt the content of training sessions, workshops or seminars to fit the 
concerns of the teachers involved, have a much better chancs of changing educational practice 
than activities which are not designed in this way. 

Other research bindings which informed our seminar design showed that teachers are more 
likely to use new content and teaching approaches in their classroom if they see specific 
instructional approaches modeled in training sessions; have time to practice the approaches 
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modeled, and adapt the materials to their own situations; iearn substantially more content than 
they plan to teach, and engage in learning experiences at their own level, while preparing to use 
a new approach with children; have opportunities to reflect on their own experiences, talk 
things over with colleagues, and experience support over time; and are recognized for their own 
areas of expertise. 

We began our seminar series with a group of twelve teacher/researchers: a team of two teachers 
and a computer coordinator from each of four Boston area cities and towns. We met for eleven 
sessions, three all-day meetings and eight after-school meetings from 4:00 - 6:30 PM. Lunches 
were provided for the all-day sessions, and the teachers and project staff took turns 
contributing refreshments for the after-school meetings. 

Each session began with a review and revision of the daily agenda, and ended with a four-question 
evaluation form. Each session was recorded, and parts of the recordings were transcribed for 
inclusion in the Notes and Commentarv . compiled for each session by the project staff. 

Activities varied from session to session. For example, the agenda for Session #3 (February 25, 
1987) shows that the group started with refreshments; broke into triads to discuss "What do we 
look foi when we observe students learning with Logo?"; reported back to the larger group; 
then broke into school-district teams to choose the two particular students whose work each 
teacher would follow for the rest of the year. The agenda for Session #6 (April 8, 1 987) shows 
that the group started with refreshments; broke into two research groups focussing on examples 
of student Logo work brought in by teachers, each with a chairperson and a presenter; went on 
to share details about conference participation and presentation planning; and ended with a 
mathematical challenge for the researchers to solve in Logo. 

The research seminars evolved through four distinct phases over the course of eleven sessions: 

1 . Learning to see: The first few weeks of the project were devoted to finding out what we were 
looking for and developing a common language to describe it. 

2. Giving more structure to the research: These sessions were devoted to identifying and 
refining specific research questions, choosing students to follow, looking at the kinds of data 
available, and structuring the data collection and observations. 

3. Looking at the patterns in the data: These sessions were devoted to attempting to understand 
what we were learning, and reflecting on our process of gathering data. 
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4. Writing project reports: At \he end of the seminar series, the teachers wrote individual 
reports on their classroom research. Throughout the project, staff members wrote ongoing 
Notes and Commentary and progress reports in preparation for writing research reports. 

We maintained a circulating library of shared resourcos: research report!., Jistrict Logo 
curriculum guides, Logo activity books and related magazines for participants to borrow. Most 
participants took advantage of this. Two members borrowed Logo research reports and most 
borrowed materials and books with Logo activities. We felt constrained by time limitations and 
did not get an annotated bibliography written for this group. We plan to continue the practice of 
sharing resources and expeut to find more ways of putting relevant materials into each person's 
hands in future seminars. 

The process we developed requires a group of committed participants, and several designated 
roles: a session leader; a note-taker; a Notes and Commentary writer; someone to copy disks 
and data brought in by participants for the group; someone to record the sessions; a librarian 
and food providers. These roles can be rotated or reassigned periodically. 

We consider the teacher research seminar to be a generic form. The specific materials and 
approaches we developed, in this case the critical aspects of Logo, could be replaced by a 
different research focus. 



111. OBSERVATION AND DATA COLLECTION TECHNIQUES 
1. Structured Interviews: 

Each of the teacher/researchers in our seminar, besides being an experienced classroom 
teacher, had a mirnmum of two years experience teaching with Logo. Before beginning the 
^^minar process, we conducted structured interviews with participants at their school sites. The 
inten/iewgave us an opportunity to visit each classroom, meet with school administrators and 
understand the contexts in which our colleagues were working. It also provided a key source of 
data for understanding any professional development that might occur during the course or me 
seminars (see Section IV below). Finally, it allowed us to understand how the seminar 
participants thought about Logo, how they taught it, what their goals and objectives were, what 
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they thought their students weie learning, and how they knew whether their students were 
learning it. 

The interview we used was designed by research consultants Roy Pea and Jan Hawkins, along 
with the project staff, to provide background information about ea^h teacher's professional 
background, Logo experience, educational ideas and values, classroom situation, and Logo 
teaching space. It was designed to evoke a description about their Logo teaching practice by 
asking, for example: Describe your Logo curriculum. What content do you want your studetus to 
learn this year (cognitive objectives)? What is it important for your students to be able to do 
(performance objectives)? What other types of behavior or attitude changes do you hope for 
among your students as a result of Logo (Social objectives)? The interviev^ went on to ask about 
teaching methods, where Logo fit into the day's schedule, the amount of Logo time spent both on 
and off computer and how else Logo work was structured. The section on teaching practices ended 
with questions asking about the teaching results: what students were learning, what 
discrepancies or surprises they had noticed between what they expected to happen and what 
actually had Happened, differences among learners and any specific problems at specific stages of 
Logo learning. 

The second section of the interview schedule asked about the Professional Development of the 
teacher/researcher. We asked how the participant began to use Logo, what workshops or 
courses had been taken, how tney assesed their own knowledge of Logo, and what goals they had 
for their own continued learning of Logo. We asked how each teacher reflected on practice: how 
they set and monitored personal goals for teaching, and how they recorded plans, projects, ioeas 
and problem areas. We also asked, "How do you improve and debug your own teaching?' We 
asked about their individual beliefs about Logo's importance and benefits for their students, how 
children learn it best and how they assess and monitor student knowledge and progress. 

The third section asked questions about accountability and reporting practices with regard to 
their Logo teaching. The fourth section asked about their past experience with research and their 
expectations, hopes and dreams for this exploratory' research project. 

2. Using Printouts of Student Logo Programs as Data: 

The fifth section of the interview was almost an aftertho' ght. We added it to the interview the 
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day the principal investigators started the classroom visits and teacher interviews. At the end of 
each interview we asked each teacher to give us a printout of a Logo program by one student and 
tell us something about the program and why they chose to show it to us. We intended to use the 
printout to start the process of documentating and assessing Logo learning. We had assumed that 
printing out examples of student procedures and pictures would be part of a Logo teacher's usual 
practice. We were surprised that nine of our twelve collaboratK'e researchers required 
technical assistance to print examples of student work. None of the researchers used printouts 
regularly to bund archives of student work, or to assess learning in order to plan teaching 
interventions. Nor did they use them as models for their students to study. In fact several of the 
teachers reported that they rarely looked at a student's programming code at all unless a student 
asked them for debugging help. 

After we demonstrated how to print Logo procedures and graphics, we continued to work wit", 
teachers to define an appro'^-'ate form for presenting eaxmples of student work to make them 
understandable for analysis of student learning. We developed a template indicating where to put 
the identification labels, dates and margins and a printout of the screen (see Figure 1). The 
procedures are printed out in order, with a procedure tree mapping the order in which each 
procedure and subprocedure is called. Then we helped participants learn how to do each of these 
steps. Later in the project, we developed some Logo tool procedures which generate procedure 
trees automatically and printout procedures in the correct calling order. (These Logo tools were 
adapted from models created by Richard Carter of Lesley College). 

3. What can printouts of students* procedures tell us about what they know? 

We used student projects as data for a group process in assessing student work. Our goal was to 
describe the Logo, mathematics and computer science ideas that the student used in a particular 
project. Eventually we evolved Four Questions for looking at student examples. First we asked, 
"What does this student know?** Then. "What is not known by this student?" Next. "What can 
you suggest to this student to solve his problem - in his way?" And finally. "Based on what this 
student has already done, what might he be ready to learn next?" 

Sometimes we looked at copies of a student's pnntout and ran the computer program. Some 
times we actually rearranged and added code to the program in an attempt to improve our 
understanding about how the student was thinking about a project, and to develop strategies for 
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coaching that student on debugging or programming difficulties. 

This process proved to be extremely fruitful for assessing the Logo knowledge a student was 
using. It often became the basis for diagnostic teaching using an emerging curriculum. When we 
studied several printouts of examples from the same student produced over a span of time, it was 
fairly straightforward to document increased Logo, mathematical and computer science 
knowledge. With practice, teachers cojid identify development and describe it in specific terms. 
For example, see Figure 2, showing \vjo examples of the work of Kathy, a fourth grader, 
rx)llected several weeks apart. 

4. Critical Aspects and Teacher Interviews: 

We used our (evolving) critical aspects checklist as a reference for studying printouts of 
student work (see Figure 3). To use the list effectively, we found it necessary to focus on one or 
two clusters of critical aspects at a lime. The whole list was just too large to use as a clear focus 
for observation. 

We experimented, and eventually found a way to use the critical aspects checklist as a structure 
for an interview. This gave teacher/researchers another way to access their own knowledge 
about what a particular student knew about Logo. (But we had to practice the process several 
times before we learned to use it effectively one teacher/researcher described a seminar 
demonstration of a preliminary attempt as "deadly"!) An example of the way we might use the 
interview is to ask the teacher what the student knows about using procedures. By looking at 
student examples we saw that that Sally and Pattie, whose keystrokes were stored in dribble 
files, probably did not know how to define procedures (see Figure 4). Kathy's show she was 
learning to combine steps and eventually define her projects in smaller conceptual pieces (see 
Figure 2). Whereas Heather shows us in her diagrams a fairly sophisticated way of using a 
superprocedure as a planning device (see Figure 5). 

We developed and used this interview form towards the end of our project, and anticipate that 
teacher/researchers will be able to use it to support each other in future seminars. We expect 
that this process can also be used by experienced teachers, working with colleagues in their own 
schools, to structure their assessments of student knowledge. 
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5. Group Discussions and Notes, and Commentary: 



In our research meetings we used discussions and group conversations as another way to support 
teacher/researchers in gaining access to their own knowledge and observations. We audiotaped 
most jroup discussions as part of the project documentation, and these tapes were used to write a 
set of Notes and Commentary on each meeting. The writer of Notes and Commentarv reflected on 
some of the key ideas, insights and concerns raised in discussions and often quoted a participant's 
exact words or program examples (Watt, M. 1979; Carini, 1979; and Schon, 1983). 
Participants reading the Notes and Commentary found that it provided another opportunity to 
reflect on their experiences and to synthesize their own knowledge. 

We also used the tapes of one meeting to help set the agenda for the next one. In this way we 
modeled some diagnostic teaching strategies within the research bominars themselves, and were 
able to work closely with the participants to support individual and group development and 
concerns. 

6. Seminar evaluation Questions: 

Another form of seminar assessment data came from the responses to the four evaluation 
questions we asked each person to answer at the end of every session : 

1. Question #1: What was the most important learning insight you had at today's meeting? 

2. Question #2: What were you most perplexed by or concerned about at today's meeting? 

3. Question #3: What would you like to talk about - or do < in future research meetings? 

4. Question #4: What help, collaboration, or resources would support your continued work? 
Answers to these questions were collated into a group response sheet and copied for everyone. 

7. A case-study process to support research: 

Another specific technique we developed is a case-study process to facilitate a 
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teacher/researcher in presenting data and research questions about a student's Logo work. Its 
design is based on previous work by Molly Watt (and others) while she was a Program Associate 
at the Prospect Archive and Research Center. North Bennington. Vermont, with Patricia Carini. 
The process was formalized, with specific roles and procedures, so that it could be used as a 
model, first for teacher/researchers in the seminar, and later for teachers to use with students 
in the classroom (Carini. 1979 and Martin, 1987). 

Here's how this case-study approach works: Suppose a teacher has brought in several examples 
of a student's work to assess and wonders about the logic in the way the student is naming 
procedures. The procedure names seem miscellaneous to her and it is the teacher's impression 
that this student is often confused about how to find his project because he can't remember its 
most recent name or the names of its component parts. She wants the group to look with her at 
the student's work, trying to make sense of whatever logic she can identify and then have several 
strategies in mind for supporting the student in finding a scheme which would serve the student 
better. 



These concerns lead to a focussing question which the teacher gives to the chairperson. (If the 
teacher does not have a clear focussing question it is the chairperson's rei,ponsibility to support 
her in finding one before meeting with the group.) The chair examines the data and makes sure 
that there is a computer for demonstration, if needed, or that printouts or journal notes are 
reproduced for the group to easily see. She then calls the meeting together, introduces the 
teacher, any description needed, the focusing question, the data and any constraints. 

The teacher/presenter then describes the data, as specifically as possible, avoiding judgemental 
statements, and allowing the meaning to emerge from the description, the process and the data. 
The chair may invite the group to add descriptions of what they notice, without seeking 
consensus or discussing interpretations. The chair facilatates. takes notes and periodically 
summarizes. Then the chair asks for responses or suggestions about the focussing question. The 
group members offer specific insights. The chair writes down each suggestion without discussion 
or evaluation or questions. The presenter listens without responding to or evaluating the ideaS 
offered. 




After the group has completed supporting the presenter's observations, and responded to her 
focussing question, the members have a brief discussion of what was interesting and what they 
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learned from the process about their own work and students. 



Later it is the cnair's task to write up notes about the meeting for everyone. 



The development o* this process was begun during our research seminars, and completed during 
the Assessing Logo Learning mini-course taught by Molly Watt and Stephen Shuller at The Logo 
Institute, August, ' 987. In future research seminars, notes on research presentations will be 
integrated into the seminar process. 



8. Individual Classroom Research Projects: 



The teacher/researchers focussed their classroom research around their own pedagogical 
questions. Some examples are: 

- How can I develop creative challenges to help students learn to use variables in Logo projects, 
which fall within my mathematics curriculum and the abilities of my students? 

- How can we get more students to use meaningful names for their subprocedures? 

- How can I get students to use superprocedures in a top-Lwwn programming style? 

Questions about teaching strategies led to specific research questions, that could be used to 
determine the types of data that each teacher wanted to collect about what their stud ''nts were 
learning. The teachers reformulated these questions several times, trying to find the right 
match between what they wanted to find out, and the data thev were able to collect naturally as 
part of their on-going work. 



Another way that teacht s focussed their research was by choosing two students to observe, 
closely documenting their Logo leaming. We asked them to choose one student about whom there 
was some particular aspect that interested them professionally. Then they chose a second student 
of the other sex, and with a different level of Logo expenise. (Our general categories were 
"strugy.wng," "average," and "facile" Logo user.) 



IV, PROFESSIONAL DEVELOPMEffT 
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Ar important outcome of our research involves documenting the professional development of the 
teachers who partic'pated with us as collaborative researchers. We have articulated a set of 
criteria for determining whether professional development occurred among participants in our 
research seminars: 

• Do teachers indicate better understanding and articulat'On of what their students are 
learning? 

• Are there any indications of changes in classroom practice during the process of this research 
project? 

• Are there indications of willingness to share work with other professionals through writing 
and conference presentations? 

The data we have collected, as a basis for analysis of professional development, include: 

• pre/post structured interviews, described above 

• pre/post Stages of Concern Questionnaires 

• session-by-session materials: teachers' journal entries; teachers' examples of student work; 
teacher's research questions - original and revised; participants' evaluation forms for each 
:>ession 

• written research reports by teachers 

• tape recordings cf oral reports by teachers 

• letters from participants about the project 

• project information forms indicating future plans of participants 

• information about presentations by participants at conferences and within their own districts 

Preliminary review of our data with regard to the above criteria gives us reason to believe that 
a full analysis will demonstrate significant professional development on the part of participating 
teachers. We expect to complete the analysis by March 1988. The two illustrations that follow 
show what we consider to be indications of professional development. 

The first example from our structured interviews with David Crump, a fourth grade teacher, 
demonstrates a change in his articulation of his approach io assessing student learning: 
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February Interview: 

Q 25: What is your belief about how to assess what students know and monitor their progress? 
A: My own experience in that area is based upon what students show me, and explain what they 
do. I have to say that I do get insight into how children are thinking when they ask me questions 
about how to do cortain things, or to see that they at least know that they need to know something 
more to do what they want to do. 

June Interview: 

Q. 25: What is your belief about how to assess what students know and monitor their progress? 
A: Over time. Seeing changes. And documenting those changes. And actually holding it up to some 
measure, whether that be the critical aspects or some other criteria. Seeing the physical 
evidence that students have been able to do some things. They've made some progress, 
Q: In order to do this, you need to keep track of that information, because otherwise you can't 
compare it. 

A: Yes. And it doesn't have to be every project or every time. It can be at the beginning of the 
yea^ in each of the next two quarters, and at the end of the year. 

The second example from the same teacher is an excerpt from his final research report and 
indicates a change in his understanding about Logo's importance in the elementary school 
curiculum as a vehicle for students learning about mathematics, problem solving, 
programming and their own thinking. 

Excerpt from **A Summary of the Classroom Research Conducted During the Logo Collaborative 
Research Project" by David Crump, June, 1987: 

Finally I will reflect upon some differences in my way of thinking about Logo that resulted from 
my participation in the LCRP, Over the past five years I have been involved in either learning 
about or teaching Logo. I honestly felt that I knew all I ever needed to know about Logo in order 
to very successfully teach Logo in my classroom as delineated by our school curriculum, 

I recently became one of those educators who began to question why so much emphasis was being 
placed on the teaching and learning of Logo in elementary schools. I wondered if the benefits 
derived from children using Logo justified the time, effort and money invested in having it 
taught. Questions also surfaced in my mind concerning my own level of un:ferstanding of and 
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knowledge about Logo, I began to feel that if ! y/as more awaro of some of the finer points of Logo, 
and understood better some of Logo's more powerful features, perhaps I might begin to see things 
in Logo that I'd been misting. When the opportunity became available to me to participate in the 
LCRP, I jumped at the chance, hoping to resolve some of the issues about Logo that concerned me. 

The intensive training in the critical aspects of Logo learning that was a component of the LCRP 
seminars by far outweighs and outdistances anything I learned about Logo in workshops or 
courscfS. The training seminars constituted not only a redefining of what is important about 
Logc, but also a refining oi Logo learning that I believe will enable me to be a much more 
effective teacher of Logo. 

What I learned about recognizing and understanding the critical aspects of Logo learning, coupled 
with teaching strategies, methods of osen/ation, as well as other material developed from this 
project related to methods of evaluating and enhancing Logo learning of students^ ic of greater 
value to me, and should be a model made available for other teachers of Logo. I believe that 
knowledge of and training in the critical aspects of Logo learning can provide a framework for 
curriculum developers who may need to reformat their present Logo curriculum, and develop 
new activities that provide the proper Logo experiences for students, so that they do progress in 
their learning and understanding of Logo, 

I am now ready to justify in my own mind why the teaching oi Logo is important If taught the 
right way, students learn to think both logically and creatively. Not only do they have the 
opportunity to learn about Math, problem solving, and programming, but more importantly they 
begin to learn about their own thinking, 
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Figure 4: Sally and Pallie's dribblo file 
(grade 5): 

0 koyslroke-by-koyslroko record of Ihoir work shows that 
Ihoy nro not liBlng procoduros at this tinio. 
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Figure 5: Heather's (grade 6) plan for Slalue 

of Liberty project: shows that she has a good 
undorstopdhig of subprocoduros, and usos a suporprocodure 
ns a plannhig dovlco. 




